There are many server-based software applications that involve latency-sensitive message traffic over a packet-switched network such as the Internet. In many of these applications, the volume of message traffic is very high. Examples of these applications are Internet telephony, single- and multi-player gaming, music sharing, and other “peer-to-peer” type applications.
An application service provider may make the software application available to many remote users. To accommodate an increase in demand for this service, service providers sometimes add more servers with the same software application to the service provider's network of servers, or server farm. Typically, each of the parallel servers has a unique network address (e.g., Internet Protocol (IP) address). In some instances, service providers install a device known as a load balancer at the “front-end” of the server farm. The load balancer typically has a unique network address, which is known and used by all clients to access the provider's services. By accessing the load balancer, users need only know one network address, that of the load balancer, to gain access to the service.
The load balancer may distribute incoming traffic among the parallel servers evenly. The load balancer may also direct requests to a specific server if necessary. A load balancer has an upper bound to the amount of traffic it can handle. For example, a typical load balancer may saturate at a rate of 800 requests per second where the data packets are relatively small in size, i.e., light-weight packets. The number of requests that a load balancer can handle is drastically reduced as the size of the data packets increases, i.e., become heavier. Heavy packets may reduce the saturation level of a load balancer to 400 requests per second. For software applications where there is a high volume of traffic from users to the server, the load balancer may become a bottleneck for message traffic. This may lead to an increase in latency. Also, directing all messages through a load balancer in some cases results in an inefficient use of resources.
Like reference symbols in the various drawings indicate like elements.